improvements to `parse_dtype` #3264

d-v-b · 2025-07-17T14:17:44Z

Add a new function parse_dtype. parse_data_type is kept around but it just wraps parse_dtype. The reason for this change is naming consistency -- the ZDType methods already use the "dtype" abbreviation extensively, so it's potentially confusing that parse_data_type does not.
Handle strings and sequences as potential json-like inputs. Adds tests to ensure that the JSON form a of a dtype is a valid argument to parse_dtype (with the exception of "|O", which is ambiguous).

closes #3263

…more JSON-like inputs, and test for round-trips

…into widen-parse-data-type

d-v-b · 2025-07-17T14:23:05Z

cc @TomNicholas

codecov · 2025-07-17T14:35:40Z

Codecov Report

Attention: Patch coverage is 80.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 59.65%. Comparing base (fd5425b) to head (8d2c0f0).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/zarr/core/dtype/__init__.py	75.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3264      +/-   ##
==========================================
+ Coverage   59.56%   59.65%   +0.08%     
==========================================
  Files          78       78              
  Lines        8684     8690       +6     
==========================================
+ Hits         5173     5184      +11     
+ Misses       3511     3506       -5

Files with missing lines	Coverage Δ
src/zarr/core/array.py	`69.02% <100.00%> (ø)`
src/zarr/dtype.py	`0.00% <ø> (ø)`
src/zarr/core/dtype/__init__.py	`30.00% <75.00%> (+5.92%)`	⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

d-v-b · 2025-07-17T15:54:50Z

d684ada adds a test to ensure that parse_dtype is the same as parse_data_type

dstansby

Nice - I like the name change. Having two identical functions in our API seems a bit confusing from a user POV: https://zarr--3264.org.readthedocs.build/en/3264/api/zarr/dtype/index.html#functions. Could you remove parse_data_type from __all__ so it's removed from the docs, but will still be imported and work for backwards compatibility?

src/zarr/core/dtype/__init__.py

src/zarr/dtype.py

Co-authored-by: David Stansby <dstansby@gmail.com>

…into widen-parse-data-type

d-v-b · 2025-07-22T10:15:37Z

a notable recent change: I widened the accepted inputs even further, and now parse_dtype / parse_data_type will accept the output of dtype.to_json(zarr_format=2), which allows you to unambiguously request a vlen string dtype for zarr v2 data via a dict like {"name": "|O", "object_codec_id": "vlen-utf8"}

src/zarr/core/dtype/__init__.py

dstansby · 2025-07-22T10:52:33Z

src/zarr/core/dtype/__init__.py

+        directly, or a JSON representation of a ZDType, or a native dtype, or a python object that
+        can be converted into a native dtype.
+    zarr_format : ZarrFormat
+        The Zarr format version. This parameter is required because this function will attempt to


A stretch goal (for another issue/PR) - would it be possible to make this optional for certain inputs (e.g., converting 'int32' doesn't depend on the version of the zarr spec)

I would rather not have a function where one parameter is optional depending on the value of another parameter. That should probably be a different function entirely, e.g. one that specifically takes numpy dtype-like inputs and finds the corresponding zarr data type.

src/zarr/core/dtype/__init__.py

Co-authored-by: David Stansby <dstansby@gmail.com>

maxrjones · 2025-07-25T00:44:24Z

I think more work here is needed for a gentle deprecation of parse_data_type in favor of parse_dtype. People using the public function will currently have a failure on the next release due to the change in import structure (e.g., virtualizarr - https://github.com/zarr-developers/VirtualiZarr/actions/runs/16510795789/job/46692311156?pr=736).

d-v-b · 2025-07-25T06:06:21Z

good catch max, looks like we need to fix the import

add parse_dtype as ergonomic replacement for parse_data_type, handle …

ed56505

…more JSON-like inputs, and test for round-trips

github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Jul 17, 2025

d-v-b added 3 commits July 17, 2025 16:19

Merge branch 'main' of https://github.com/zarr-developers/zarr-python …

ca6196d

…into widen-parse-data-type

update docs

d04557d

changelog

6009923

github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Jul 17, 2025

d-v-b requested a review from a team July 17, 2025 14:23

d-v-b changed the title ~~improvments to parse_dtype~~ improvements to parse_dtype Jul 17, 2025

d-v-b added 3 commits July 17, 2025 16:38

remove type: ignore

082ad49

Merge branch 'main' into widen-parse-data-type

b54bbe3

add test to check that parse_dtype is parse_data_type

d684ada

dstansby reviewed Jul 21, 2025

View reviewed changes

src/zarr/core/dtype/__init__.py Outdated Show resolved Hide resolved

src/zarr/core/dtype/__init__.py Show resolved Hide resolved

src/zarr/dtype.py Outdated Show resolved Hide resolved

d-v-b and others added 6 commits July 22, 2025 11:29

Update src/zarr/dtype.py

5feb937

Co-authored-by: David Stansby <dstansby@gmail.com>

fix docstring

c42edf6

Merge branch 'main' into widen-parse-data-type

d234ae2

support the output of to_json(zarr_format=2) as input to parse_dtype

6285005

Merge branch 'widen-parse-data-type' of github.com:d-v-b/zarr-python …

3a82a26

…into widen-parse-data-type

lint

b95f3be

remove infinite recursion

b0edab6

d-v-b requested a review from dstansby July 22, 2025 10:23

dstansby reviewed Jul 22, 2025

View reviewed changes

d-v-b and others added 5 commits July 22, 2025 13:22

Update src/zarr/core/dtype/__init__.py

385efa0

Co-authored-by: David Stansby <dstansby@gmail.com>

Update src/zarr/core/dtype/__init__.py

f7b5387

Co-authored-by: David Stansby <dstansby@gmail.com>

Update src/zarr/core/dtype/__init__.py

6cef7d7

Co-authored-by: David Stansby <dstansby@gmail.com>

Update src/zarr/core/dtype/__init__.py

9a7d1db

Co-authored-by: David Stansby <dstansby@gmail.com>

Update src/zarr/core/dtype/__init__.py

d741513

Co-authored-by: David Stansby <dstansby@gmail.com>

d-v-b and others added 2 commits July 22, 2025 13:27

Update src/zarr/core/dtype/__init__.py

8585158

Co-authored-by: David Stansby <dstansby@gmail.com>

Merge branch 'main' into widen-parse-data-type

8d2c0f0

dstansby approved these changes Jul 22, 2025

View reviewed changes

dstansby merged commit a27d4d6 into zarr-developers:main Jul 22, 2025
30 checks passed

d-v-b deleted the widen-parse-data-type branch July 22, 2025 12:16

maxrjones mentioned this pull request Jul 25, 2025

Run slow tests in ci based on test-slow label zarr-developers/VirtualiZarr#736

Open

7 tasks

dstansby mentioned this pull request Jul 25, 2025

Put back parse_data_type import #3292

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

improvements to `parse_dtype` #3264

improvements to `parse_dtype` #3264

d-v-b commented Jul 17, 2025

Uh oh!

d-v-b commented Jul 17, 2025

Uh oh!

codecov bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

d-v-b commented Jul 17, 2025

Uh oh!

dstansby left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

d-v-b commented Jul 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dstansby Jul 22, 2025

Uh oh!

d-v-b Jul 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxrjones commented Jul 25, 2025

Uh oh!

d-v-b commented Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

improvements to parse_dtype #3264

improvements to parse_dtype #3264

Conversation

d-v-b commented Jul 17, 2025

Uh oh!

d-v-b commented Jul 17, 2025

Uh oh!

codecov bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

d-v-b commented Jul 17, 2025

Uh oh!

dstansby left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

d-v-b commented Jul 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dstansby Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

d-v-b Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxrjones commented Jul 25, 2025

Uh oh!

d-v-b commented Jul 25, 2025

Uh oh!

Uh oh!

improvements to `parse_dtype` #3264

improvements to `parse_dtype` #3264

codecov bot commented Jul 17, 2025 •

edited

Loading